Refresh the UK Universal Credit target contract - #727
Conversation
a68d686 to
051c9ab
Compare
|
Adversarial review completed and fixes pushed in Findings fixed:
Verification:
GitHub CI is queued on the new head. |
vahid-ahmadi
left a comment
vahid-ahmadi
left a comment
There was a problem hiding this comment.
Reviewed as the first link in the chain that closes the UC row on #731, so I checked the seam to chronicle#184 rather than just this diff in isolation. The accounting closes (652 = 610 + 42, 187 targets, 184 mapped), the v1_56_16_refresh note correctly distinguishes "added registry row" from "deleted dispatch names that were never registry rows," and the targeted suites pass for me on the rebased head.
@juaristi22's catch on the binding is the one that mattered — value_variable: universal_credit under a benefit-unit count declaration would have summed UC amounts while the contract said "count units," and it would have done so silently at activation rather than failing. benunit_count + benunit + the focused regression test is right, and the note distinguishing DWP's "household" (UC assessment unit) from an ONS household is exactly the disambiguation uk-data#457 exists over.
One design question before this activates: which month becomes the target value?
chronicle#184 emits eleven monthly benefit-unit counts, and this target's selector pins only source_name + source_concept, so it matches all of them. The resolver's behavior for that case is _latest_period_selector_match — filter to facts at or before the reference period, then take the latest — so the bound value silently becomes a single month's snapshot, whichever month happens to sit closest to the target period, with an ambiguity raise only on ties.
For a stock measure that is defensible, but it should be a declared choice rather than an emergent one, because the two failure modes are opposite and both quiet:
- If the monthly facts fall at or before the reference period, the target binds one month. UC caseload trends and has seasonality, so "latest month ≤ period" is a materially different number from an annual mean — and #731's UC row is a caseload gap (per-unit entitlement already matches admin to £11.3k), so the number this target binds is precisely the thing being calibrated to.
- If they all fall after it (monthly DWP releases usually run ahead of a 2023 reference year),
_eligible_selector_matchesempties and the reference raises "did not match a Ledger fact at or before target period" — a fail-closed refusal, which is the good outcome, but it means activation will surface as an error at #622 wiring rather than here.
Cheapest fix is to make the intent explicit in the declaration — a period pin or a dimension_values pin naming the reference month, or an annual-average concept on the chronicle side if that's the intended basis. Worth settling now: this is the seam where "the target is 6.7m UC units" turns into a specific number, and a silently-chosen month is the kind of thing that gets discovered later as an unexplained calibration residual.
Smaller: the selector uses key source_concept with the value dwp.uc_benefit_units, which is the chronicle fact's measure.concept — its source_concept is dwp.uc_households. This resolves correctly today only because the source_concept key's candidate tuple falls back to the primary concept, so both spellings match. It works, but the pin reads as if it names the source-side spelling when it names the canonical one; pinning dwp.uc_households (or using a concept-named key) would make the intent legible and wouldn't depend on the fallback arm staying tolerant.
Nothing blocking — the contract-side declaration is correct as written, and both points are about what happens when it activates.
|
Followed the seam through to chronicle#184's actual periods, which settles which branch of the question above obtains — both of them, in sequence: The DWP facts span 2025-03 through 2026-02. Against the current UK activation period of 2023 (the membership rule in #707), Then at the #723 retarget (FRS 2024-25, calibration year 2025) the same selector goes live and eligible becomes 2025-03…2025-12, so latest-wins binds December 2025 alone as the UC caseload target. That is the specific number that would drive the calibration correcting a −52% caseload gap, chosen by a tie-break rule rather than declared — and December is exactly where you would want to know whether a seasonal peak or trough is being picked up as the annual level. So the period pin I suggested isn't cosmetic: without it the target is inert now and implicitly December-anchored later, and neither state is visible from the declaration. Same fix as before — pin the reference month explicitly, or add an annual-average concept on the chronicle side if the intended basis is a year rather than a point-in-time stock. |
051c9ab to
7ff3f16
Compare
|
Follow-up: the first post-push CI run exposed one more golden tied to the contract identity digest:
Re-verified locally:
Pushed the amended fix as |
Rebasing over merged #727 (and #730 beneath it): main's TestUnevidencedArms now enumerates the closed set of gates that report evidence_absent when the battery runs unarmed, so the new uk_calibration_reference_coverage gate joins that set. Also squares the indentation of the earlier enumeration update in the national-build terminal-batch test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
3 participants
Summary
Part of #622 and the UC slice of #701. Refreshes the UK national calibration contract from policyengine-uk-data
ebf733cto 1.56.16 (12a1e028afeef08d8b2d74ee03fd9de3a78b2dd3), following policyengine-uk-data#452 and chronicle#184.The contract adds the value-free
dwp.uc.householdsdeclaration: a prepared UC-receipt indicator summed at benefit-unit level. DWP “household” means the UC unit of assessment, not an ONS household.This PR deliberately stops after the contract refresh. It does not start #623's calibration step and does not add an active UC reference, for the period-semantics reason below.
Period-semantics receipts and verdict
packages/microcosm-build/src/microcosm/build/ledger_targets.py:48-68: eachLedgerTargetReferenceowns an optional per-referenceperiod; periods are not restricted to one file-wide value.packages/microcosm-build/src/microcosm/build/ledger_targets.py:797-845: selector resolution filters to facts at or before that reference's period and fails loudly when no eligible fact exists.packages/microcosm-build/src/microcosm/build/country_spec.py:804-817and:925-952:target_references.jsonis loaded and typed intoCountrySpec.target_referencesreceipt inpackages/microcosm-build/src/microcosm/build/uk_runtime/national_build.py: the national build does not evaluate this resource yet.Verdict: references support per-reference periods in the Belgian convention, but the present UK national build evaluates none of them. A 2025 UC fact cannot resolve for the certified 2023 active subset because the resolver admits only facts at or before the reference period. Labelling the UC row 2023 would fail closed; labelling it 2025 would create an inert mixed-period declaration rather than a target evaluated by the national build. Per the requested stop rule, activation is deferred instead of forcing a misleading period.
Consequently, fixture-backed activation and its loud-failure test are also deferred: adding tests for an activation that the verdict disallows would contradict the stop rule. The Chronicle bundle dependency remains chronicle#184 and a later released consumer feed.
Registry-parity accounting
The governed surface changes from 651 rows (609 mapped + 42 signed exclusions) to 652 rows (610 mapped + 42 signed exclusions):
dwp/uc/households→dwp.uc.households;obr/jobseekers_allowanceremains because it is the distinct published JSA expenditure target, not one of those deleted UC split names.The exact closure invariant remains 187 declarations = 184 distinct mapped target IDs + 3 signed unmapped declarations.
Tests
Commands:
Scope
No calibration, dataset build, upload, merge, or active-reference change is included. This PR does not close #622; other items remain. The separate #623 claim remains for a later run.